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ABSTRACT 



Multispectral imaging apparatus for producing a com- 
posite dual wavelength visible/infrared image includes 
a wavelength independent focus reflective optical sys- 
tem for receiving a visible and infrared radiation from a 
scene and for providing a focused beam of radiation, a 
beam splitter for splitting the beam into a beam of visi- 
ble radiation and a beam of infrared radiation, a visible 
radiation detector for receiving the visible radiation and 
for producing a first signal, and infrared radiation detec- 
tor for receiving the infrared radiation and for produc- 
ing a second signal, and a signal processing and display 
system for converting the first and second signals into 
video signals corresponding to a visible black and white 
image of the scene and a step-tone or false color infrared 
image of the scene. The images are combined in exact 
spatial registration to produce a composite image com- 
prising a black and white visible image with portions of 
the image highlighted in color in accordance with the 
infrared radiation received from corresponding por- 
tions of the scene. 

17 Claims, 1 Drawing Sheet 
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COMPOSITE VISIBLE/THERMAL-INFRARED 
IMAGING APPARATUS 

BACKGROUND OF THE INVENTION 5 

This invention relates generally to multispectral im- 
aging systems, and more particularly, to a dual wave- 
length band imaging system for producing a composite 
visible/thennal*infrared image of an object or a scene. 

Although the human eye is responsive only to radia- 10 
tion concentrated in the relatively narrow visible light 
region of the electromagnetic spectrum and is capable 
of recognizing an object from its reflected light even 
under poor lighting conditions, all objects reflect or 
emit nonvisible radiation which is useful for identifying 15 
or otherwise characterizing the objects. For example, 
passive infrared (IR>thermal imaging (thermography) 
utilizes the passively emitted thermal radiation from an 
object as for non-destructive testing, medical diagnosis, 
surveillance and military target identification, and guid- 20 
ance and tracking. However, it is difficult to recognize 
even familiar objects from images produced from the 
object's passively emitted thermal radiation in the infra- 
red spectral region in the absence of familiar visible 
cues. Under the pressures of simulated or actual com- 25 
bat, even experienced pilots have been known to fire at 
a "hot" feature revealed by their FLIR (Forward Look- 
ing Infrared) display system without being able to iden- 
tify the target Even the highest resolution thermal 
imaging systems can do little to improve this situation 30 
because of the fundamental differences in the behavior 
of matter in the visible and infrared spectral regions. 
For example, glass and water are transparent to the 
human eye, but are virtually opaque in the thermal 
infrared region. Paints and semiconductors are visually 35 
opaque, but are partially transparent to infrared radia- 
tion. A further complication is that all matter glows to 
some extent at infrared wavelengths, and an infrared 
image is, in effect, a radiant temperature map of an 
object or scene. 40 

Multispectral imaging systems which produce a com- 
posite visible/infrared image are known. Such systems 
have the advantage of combining in a single image both 
visible and thermal information and are useful in en- 
abling thermal features in a complex scene to be posi- 45 
tively located and identified. Previously known sys- 
tems, however, suffer from several problems, a princi- 
ple one being the difficulty in obtaining exact spatial 
registration of the visible and infrared images. In sys- 
tems which employ separate optical, detection, and 50 
image processing and display systems for the visible and 
thermal-infrared spectral bands, some degree of paral- 
lax is unavoidable due. mainly to the separate optical 
trains. While overlaid multispectral images may be 
formed using digital image processing techniques, it is 55 
difficult to achieve exact spatial registration in current 
digital image processing systems. Also, digital process- 
ing of high resolution images is usually not performed in 
real time because of the extensive computations re- 
quired. Even systems which have a common optical 60 
train have difficulty in achieving spatial registration of 
the visible and infrared images due to the difficulty in 
focusing the widely separated visible and infrared 
wavelength bands. Thus, it is desirable to avoid these 
problems in a composite dual wavelength visible/ther- 65 
mal imaging system. 

In U.S. Pat. No. 4,679,068 issued July 7, 1987, and 
assigned to the same assignee as this patent application, 
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there is described and claimed a composite visible/ther- 
mal infrared imaging system. This multispectral imag- 
ing apparatus includes a hybrid visible/infrared radia- 
tion detector which is a common focal plane onto 
which is focused both the visible and the infrared radia- 
tion from a scene to provide separate signals to the 
video imaging portions of the system. These signals are 
converted to a visible black and white image of the 
scene and a step-tone false color infrared image of the 
scene which images are combined in exact spatial regis- 
tration to produce a composite image. 

It is further desirable to provide an improved com- 
posite dual wavelength visible/thermal infrared imag- 
ing system over that described in the above discussed 
U.S. Pat. No. 4,679,068 wherein the visible and infrared 
radiation from a scene are focused at first and second 
focal planes, respectively, which are located the same 
distance from the optical system and thus, allows the 
user a wider choice of visible or infrared detection 
means. 

SUMMARY OF THE INVENTION 

The invention affords an imaging system which pro- 
vides a composite visible image and a thermal-infrared 
image of a scene with the visible and infrared images in 
exact spatial registration. The composite image may be 
displayed on a conventional color motor as, for exam- 
ple, a high resolution black and white visible image with 
a false color step-tone or saturated white overlay of 
infrared highlights. Moreover, the invention enables 
control of the infrared image so that only infrared fea- 
tures having radiances within a preselected range are 
displayed in order to avoid a cluttered composite image. 
In addition, different discrete ranges of radiance values 
may be displayed as different colors, which enables 
selected thermal features in a complex scene to be posi- 
tively located and identified. 

Briefly stated, in accordance with one aspect, the 
invention provides multispectral imaging apparatus 
comprising a wavelength-independent focus reflective 
optical system for receiving visible and infrared radia- 
tion from a scene and for focusing the visible and infra- 
red radiation at first and second focal planes, respec- 
tively, which are located the same distance from the 
optical system; first detector means disposed at the first 
focal plane for providing a first signal representative of 
the visible radiation; second detector means disposed at 
the second focal plane for providing a second signal 
representative of the infrared radiation; and means re- 
sponsive to the first and second signals for displaying a 
composite image of the scene, the composite image 
comprising a black and white visible image of the scene 
with portions of the image highlighted in color in accor- 
dance with the infrared radiation received from corre- 
sponding portions of the scene. 

In another aspect, the invention provides multispec- 
tral imaging apparatus comprising a non-refractive 
optical system for receiving visible and infrared radia- 
tion from a scene and for_pr6viding a focused beam of 
combined visible radiation and infrared radiation; 
means for splitting the focused beam into a first beam of' 
visible radiation and a second beam of infrared radia- 
tion; first detector means for receiving the first beam 
and for providing a first signal representative of the 
visible radiation; and second detector means for receiv- 
ing the second beam and for providing a second signal 
representative of the infrared radiation; and means re- 
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sponsive to the first and second signals for displaying a Video frame generator 34 preferably includes an 

composite image of the scene, the composite image adjustable lower threshold control 36 which enables 

comprising a black and white visible image of the scene adjustment of the lower sensitivity threshold of the 

with portions of the image highlighted in color in accor- infrared channel to a preselected value. This control 

dance with the infrared radiation received from corre- 5 affords an adjustable background threshold and ad van-, 

spending portions of the scene. tageously allows low ambient temperature objects to be 

BRIEF DESCRIPTION OF THE DRAWING eliminated from the infrared display. Thus, only objects 

or portions of a scene having temperatures greater than 

The sole FIGURE is a diagrammatic view of com- a preselected value, Le., producing infrared radiation 

posite multispectral visible/thermal-infrared imaging 10 greater than a preselected radiance value, are displayed, 

apparatus in accordance with the invention. This is convenient for avoiding cluttering of the result- 

DESCRIPTION OF THE PREFERRED m & composite display with low level background infor- 

EMBODIMENT mation. Video frame generator 34 may also include a 

second upper threshold control 38 for setting an upper 

The invention is particularly well adapted to provid- 15 cut<) ff i eV el so that objects or portions of the scene 

ing a display of a composite visible/infrared image of an having temperatures greater than other preselected 

object or scene comprising a black and white visible valuc ^ ^ eliminated from the display. The two 

image which is highlighted in color with infrared infor- controls 36 and 33 thus afford an adjustable window 

mation, and will be described in that context. However, whjch be ^ ^ t hat only infrared signals within the 

as will become apparent from the description which 20 preseIected range ^twecn the u?pcT and lower thresh- 

follows, this is illustrative of only one utility of the M limjts m displaye<it k convenient for enabling 

invention. „ _ T __ ' ... , , . . objects or portions of a scene having temperatures 

Msho^mtheFIGiniE, andas described in within pre 7 etermined temperature range to be high- 

more detad shortly, comrxssite visible/Aennal-mfrared lighted on the display and to be easily identified. If 

unagmg apparatus in accordance with ^invention 25 d ^ d> mdtiple ^ J hold controU ^/be included t0 

may generally ■compose an optical system 10 for receiv- m £ adjustable wfa| ^ t0 ^ ^b- 

ing visible and infrared radiation from an object or a j. . h 

scene and for providing a focused beam of radiation 1 Jl ' A _ , ■ . - . . _ 
which is split into a first beam of visible radiation and a ™ e ^ from video frame generator 
second beam of infrared radiation. The visible radiation 30 34 * f. u PP h ^ *\ a conventional color synthesizer 
is supplied to a first detector 12 which outputs a first or color sheer 40 which converts different discrete 
signal corresponding to the visible radiation impinging of Ae Vld f e ? m }° ^rent colors to pro- 
upon the detector, and the infrared radiation is supplied *ice a steptone false color (or saturated white) video 
to a second detector 14 which outputs a second signal sl S^ corresponding to a color image of all objects 
corresponding to the infrared radiation impinging upon 35 W1 f thm the fiel d of ™* of < he op ? cal systcm havmg 
the second detector. The first and second signals are infi » re f l densities within the preselected range set by 
supplied via respective channels 16 and 18 to a signal the adjustable threshold controls 36 and 38. The color 
processing and display system 20 which processes the wdeo s i S*™ from color synthesizer may be supplied 
signals and produces a composite visible/mermal-infra- to vidco 32 » whcrc ** is combined : with the B/W 
red image of the scene within the field of view of the 40 f Tom video frame generator 30 to produce 
optical system. The composite image may be displayed a composite image on color monitor 22. The composite 
- on a conventional color monitor 22, for example, and comprises a black and white visible image which 
preferably comprises a black and white (B/W) visible » overlaid by a step-tone false color (or saturated white) 
image of the scene with portions of the visible image thermal-infrared image. For reasons which will be de- 
highlighted in color in accordance with the infrared 45 scribed shortly, the black and white visible image and 
radiation received from corresponding portions of the the step-tone false color infrared image are in exact 
scene, spatial registration. The composite image thus com- 
As shown in the FIGURE, the output signals from P^ses » visible image of the scene or object which is 
detector 12, which correspond to the visible radiation highlighted by infrared features with radiances between 
impinging upon the detector, may be supplied via chan- 50 tnc preselected upper and lower threshold limits. Of 
nel 16 to a first video frame generator 30 which formats course, either image may be separately displayed, and 
the raw signals from the detector into gray scale black the video signals may be supplied to the other devices as 
and white (B/W) image frames and outputs a corre- desired. 

spending B/W video signal. The video output signal The signal processing and display system 20 compris- 
from video frame generator 30 may be supplied to a 55 ing the video frame generators, the color synthesizer, 
conventional video mixer 32, the output of which is the video mixer, and the color monitor may be imple- 
supplied to the color monitor 22. The video signal from mented with standard commercially available devices, 
video frame generator 30 is displayed on the color mon- Both video frame generators, for example, may be sub- 
itor as a B/W visible image of the scene within the field stantially similar and may comprise any commercially 
of view of the optical system Similarly, the signals from 60 available device capable of producing standard U.S. 
detector 14, which correspond to the infrared radiation NTSC or European Pal TV video frame signals. The 
impinging upon the detector, may be supplied via chan- threshold controls 36 and 38 of video frame generator 
nel 18 to a second video frame generator 34 which 34 may simply comprise voltage threshold controls 
converts the signals into frames of gray scale black and arranged such that only signals within the preselected 
white video signals which are synchronized with the 65 threshold range are converted to a video output. Simi- 
video signals from video frame generator 30. For this larly, color synthesizer 40 may be a commerically avail- 
purpose, video frame generators 30 and 34 may be able device for converting a gray scale black and white 
driven by a common clock (CLK). video signal into a color video signal. 
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Optical system 10, which receives the visible and 
infrared radiation and provides a focused beam of radia- 
tion, comprises a wavelength-independent, non-catadi- 
optric, non-refractive reflective optical system. As indi- 
cated in the FIGURE, the optical system preferably 5 
includes a reflective Cassegrain telescope, which is 
advantageous for affording compactness, comprising a 
cylindrical housing 50 having one end 52 which is open 
to provide an entrance for the radiation and an opposite 
end 54 which is closed except for a small diameter cen- 10 
trally located exit aperture 56. A cylindrical tubular 
member 59 may extend inwardly a short distance into 
the housing from exit aperture 56, as shown. An annular 
concave primary mirror 60 may be disposed within 
cylindrical housing 50 adjacent to closed end 54 with 15 
the tubular member 5S extending through an opening 62 
in the mirror. A convex secondary mirror 64 may be 
supported at an intermediate location along the axis of 
the bousing by a support structure 66. 

Visible and infrared radiation (indicated by the dotted 20 
lines) entering opening 52 is reflected by the concave 
primary mirror 60 onto the secondary mirror 64, which 
in turn reflects the radiation so that it exits the housing 
through tubular member 58 and aperture 56 as a com- 
posite beam of visible and infrared radiation which is 25 
focused at a predetermined distance from the secondary 
mirror. The telescope may be focused by a mechanical 
focusing system (not illustrated) which moves both the 
primary and secondary mirrors. The primary and sec- 
ondary mirrors are preferably coated with aluminum, 30 
silver, gold or other suitable material which has opti- 
mum reflection characteristics at both visible and infra- 
red wavelengths, preferably throughout the 0.4-14 mi- 
crometer wavelength range. 

The focused beam of radiation from the telescope 35 
may be split into a first beam 70 of visible radiation and 
a second beam 72 of infrared radiation by a silicon beam 
splitter 74 comprising a thin dichroic semiconductor, 
such as silicon, mirror. The silicon mirror is transparent 
at infrared wavelengths longer than approximately 2 40 
micrometers and is reflecting at visible wavelengths. 
The beam splitter, 74, may also be fabricated out of 
semiconducting materials other than silicon to change 
the wavelength limits of either the infrared or visible 
channel. A gallium arsenide beam-splitter, for example, 45 
becomes transparent at approximately 0.9 microns 
wavelength. Accordingly, the infrared radiation passes 
through the mirror and is focused at a focal plane 76 at 
which the infrared detector 14 is located. The visible 
radiation is reflected by mirror 74 to a fully reflecting 50 
imaging erecting mirror 78 which reflects the focused 
visible radiation to a focal plane 80 at which the visible 
radiation detector 12 is located. Focal planes 76 and 80 
are located the same distance from the secondary mir- 
ror 64 of the telescope, and this distance corresponds to 55 
the focal point of the telescope. The image erecting 
mirror 78 for the visible radiation cancels the mirror 
image reversal caused by the beam splitter 74 so that the 
image size and orientation is identical at both the infra- 
red and visible focal planes 76 and 80, thus insuring 60 
exact image registration in both wavelength channels. 

Since the optical system is non-refractive, its focus is 
wavelength independent. Because of the widely sepa- 
rated visible and infrared wavelength bands and the 
necessity of providing a common focal plane distance 65 
for both the visible and infrared radiation, catadioptric 
Gens/mirror) and refractive optical systems are not 
suitable for the composite imaging system since the 



visible and infrared radiation would not be focused 
precisely at the same focal plane distance, which would 
cause problems in producing high resolution images 
that are in spatial registration. 

Visible radiation detector 12 may comprise any con- 
ventional imaging detector of visible light, such as a 
solid state CCD or CID detector array, or a vidicon 
tube. Visible radiation detector 12 may also comprise 
any conventional imaging detector of visible light 
whose low light level response characteristics have 
been augmented by the addition of an eletro-optic 
image intensifxer device. Also, visible radiation detector 
12 and video frame generator 30 may comprise a televi- 
sion camera without the normal imaging lens. Infrared 
detector 14 may be a conventional focal plane infrared 
detector array, and may comprise, for example, a me- 
chanically scanned infrared imaging radiometer with- 
out the normal imaging lens. For optimum infrared 
sensitivity, detector 14 is preferably operated at cryo- 
genic temperatures, preferably at the order of 77* K. 
Accordingly, the detector may be disposed within a 
liquid nitrogen dewar 90 havin g infrared transparent 
windows 92 f or admitting radiation. Alternatively, 
other cooling schemes, such as thermoelectric or Joule- 
Thompson refrigerators, may be employed for cooling 
the detector. 

While a preferred embodiment of the invention has 
been shown and described, it will be appreciated by 
those skilled in the art that changes may be made in this 
embodiment without departing from the principles and 
spirit of the invention, the scope of which is defined in 
the appended claims. 

What is claimed is; 

1. Multispectral imaging apparatus comprising a 
wavelength independent focus reflecting optical system 
for receiving visible and infrared radiation from a scene 
and for focusing the visible and infrared radiation at 
respective first and second focal planes which are lo- 
cated the same distance from the optical system; first 
detector means disposed at the first focal plane for re- 
ceiving the visible radiation and for providing a first 
signal corresponding thereto; second detector means 
disposed at the second focal plane for receiving the 
infrared radiation and for providing a second signal 
corresponding thereto; and means responsive to the first 
and second signals for displaying a composite image of 
the scene, the composite image comprising a black and 
white visible image of the scene with portions of the 
image highlighted in color in accordance with the infra- 
red radiation received from corresponding portions of 
the scene. 

2. The apparatus of claim 1, wherein the optical sys- 
tem is non-refractive. 

3. The apparatus of claim 1, wherein the optical sys- 
tem comprises a reflective Cassegrain telescope having 
movable primary and secondary mirrors for focusing 
the radiation. 

4. The apparatus of claim 1, wherein the optical sys- 
tem comprises means for splitting the radiation into a 
first beam of visible radiation and a second beam of 
infrared radiation, and means for directing the first and 
second beams onto the first and second detector means, 
respectively. 

5. The apparatus of claim 4, wherein the beam split- 
ting means comprising a thin dichroic semiconductor 
mirror which is transparent to the infrared radiation and 
which reflects the visible radiation, and wherein the 
directing means comprises a fully reflecting image 
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erecting mirror positioned to reflect the visible radia- 
tion reflected from the mirror to the first detector 
means. 

6. The apparatus of claim 1, wherein the displaying 
means comprises first means for converting the first 
signal into a first video signal corresponding to a black 
and white visible image of the scene, second means for 
converting the second signal into a second video signal 
corresponding to a color infrared image of the scene, 
third means for combining the first and second video 
signals to produce a composite video signal, and a color 
display for receiving the composite video signal and for 
displaying said composite image. 

7. The apparatus of claim 6, wherein the second con- 
verting means includes means for setting one or more 
preselected threshold levels such that only a second 
signal which corresponds to infrared radiation with a 
preselected intensity range is converted to the second 
video signal. 

8. The apparatus of claim 6, wherein the second con- 
verting means comprises means for converting discrete 
ranges of the second signal into different colors. 

9. Multispectral imaging apparatus comprising a non- 
refractive optical system for receiving visible and infra- 
red radiation from a scene and for providing a beam of 
combined visible and infrared radiation which is fo- 
cused at a predetennined distance from the optical sys- 
tem; means for splitting the focused beam into a first 
beam of visible radiation and a second beam of infrared 
radiation; first detector means for receiving the first 
beam and for providing a first signal corresponding to 
the visible radiation; second detector means for receiv- 
ing the second beam and for providing a second signal 
corresponding to the infrared radiation; and means re- 
sponsive to the first and second signals for displaying a 
composite image of the scene, the composite image 
comprising a black and white visible image of the scene 
with portions of the image highlighted in color in accor- 
dance with the infrared radiation received from corre- 
sponding portions of the scene. 



10. The apparatus of claim 9, wherein the optical 
system comprises a wavelength independent focus opti- 
cal telescope. 

11. The apparatus of claim 10, wherein the optical 
5 telescope comprises a reflective Casscgrain telescope 

having movable primary and secondary mirrors for 
focusing. 

12. The apparatus of claim 9, wherein the first and 
second detector means are located at respective first 

10 and second focal planes, and the first and second focal 
planes are each located at said predetennined distance 
from the optical system 

13. The apparatus of claim 9, wherein said splitting 
means comprises a thin dichroic semiconductor beam 

15 splitter formed to pass the infrared radiation and to 
reflect the visible radiation, and a fully reflecting mirror 
for receiving the visible radiation from the beam splitter 
and for reflecting the visible radiation to the first detec- 
tor means. 

20 14. The apparatus of claim 9, wherein the displaying 
means comprises means for converting the first signal 
into a first video signal corresponding to said black and 
white visible image of the scene, second means for con- 
verting the second signal into a second video signal 
25 corresponding to a color infrared image of the scene, 
third means for combining the first and second video 
signals to produce a composite video signal, and color 
display means for receiving the composite video signal 
and for displaying said composite image. 
30 15. The apparatus of claim 14, wherein the second 
converting means includes means for setting a prese- 
lected range of levels such that only a second signal 
which corresponds to infrared radiation within the pre- 
selected range is converted to the second video signal. 
35 16. The apparatus of claim 15, wherein the second 
converting means comprises a video frame generator, 
and the setting means comprises voltage threshold 
means for setting upper and lower threshold levels, the 
threshold means being arranged such that only a por- 
40 tion of the second signal between said upper and lower 
levels is converted to said video signal. 

17. The apparatus of claim 14, wherein the second 
converting means comprises means for converting dis- 
crete ranges of the second signal into different colors. 
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